Neural Mechanisms for Training Autonomous Robots

Minimalist neural mechanisms are suitable tools for programming and training autonomous robots. This paper explores the limitations of hand-crafted minimalist robot control mechanisms based on a neural paradigm, and then shows that these mechanisms are well suited to robot training using well understood neural learning mechanisms. Training a robot is more powerful than other methods more commonly used for robot learning (such as reinforcement learning and genetic techniques). A trained robot is told more than whether it was wrong or right for a particular action or sequence (reinforcement learning), the robot is also told what it should have done (supervised learning). Robots can hence develop appropriate behaviour much more rapidly. The neural mechanisms and training techniques have been developed on a kinematically realistic simulator. The mechanisms have been ported from the simulated vehicles to a real vision guided robot: CORGI [8]. Results from the simulation and CORGI are presented. 1 Making a Robot Behave It is difficult to make an autonomous robot perform reliably. Making a robot behave is a problem that has been approached in two distinct ways: • robot programming where the robot is told what to do, or by • robot learning where the robot is asked to determine appropriate behaviour by interacting with the environment. Programming is complicated by the need to cater for novel situations. Complex sensor spaces and internal state spaces make reliable programming troublesome. The alternative, autonomous robot learning, is still a relatively young field. Work with evolutionary computing and reinforcement learning has provided solutions to simple tasks (see [2] for an overview), but it is apparent that these approaches do not present immediate solutions for the kind of things we would like robots to do. Training a robot, on the other hand, involves placing the robot in a typical situation and showing the robot appropriate behaviour. Trained neural mechanisms have presented solutions to some real world tasks (driving a vehicle [5], for example) through training, also called supervised learning. For example, driving a vehicle was taught by having a neural mechanism learn an association between visual input and steering angles while the vehicle was under human control. When released from human control, it used the previous examples to steer through new terrain. This kind of efficient learning would be difficult to implement using a reinforcement learning or evolutionary paradigm. The tasks investigated for robot training in this paper are centred about a class of problems we call hunt and gather robotics. The type of tasks that a hunt and gather robot might perform include picking up around the house, collecting litter from streets, parks and waterways, picking crops from a field or orchard, or collecting rocks on interplanetary exploration missions. These tasks are characterised by targets and obstacles that have undefined locations and that may be static or dynamic. The task of the robot is to collect the targets while avoiding the obstacles. The nature of this task allows operation of the robot based on a reactive control system. A commonly used definition of a reactive system is a system that contains no state (or memory). A lack of internal state in a robot enhances the robot’s ability to react to dynamic situations and relieves issues in sensor and effector calibration. A reactive robot circumvents the difficult problems of knowledge representation in an unstructured environment. On the down side, a reactive robot may be less than optimal; for example, a robot may explore a region repeatedly without venturing into new areas, as it has no memory of visiting that region before. This paper will show that small neural network systems are capable of providing reactive control for a hunt and gather robot. These control systems are based on Braitenberg vehicles [1]. Simulation studies will be used to show the type of neural structures that are useful for autonomous robot behaviour generation, and then show how such structures may be generated by training. The results of the simulation study form the basis of tests on a real hunt and gather robot: CORGI.